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Abstract 

This  paper  attempts  to  identify  and  diseuss  some  of  the  major  ehallenges  to 
eondueting  patient  safety  researeh  in  medieal  group  praetiees.  First,  we  identify 
the  most  important  attributes  of  medieal  group  praetiees  to  be  eonsidered  in  this 
type  of  researeh.  The  measurement  and  speeifieation  of  these  variables  are 
diseussed,  and  the  problems  assoeiated  with  past  researeh  are  noted.  Alternate 
ways  of  speeifying  key  variables  are  discussed,  and  examples  of  methods  that 
have  been  shown  to  be  effective  are  provided.  We  then  propose  a  definition  of 
patient  safety  and  discuss  issues  around  its  measurement.  Finally,  we  discuss  how 
the  data  requirements  for  this  type  of  research  are  very  complex  and  outline 
benefits  and  challenges  related  to  using  certain  types  of  data. 


Introduction 

There  is  wide  agreement  that  many,  if  not  most,  patient  care  adverse  events 
originate  in  the  ambulatory  care  sector.  The  magnitude  of  these  events  is  just 
now  becoming  evident.  A  recent  study  of  prescription  drug  errors  found  that  13.5 
percent  of  the  prescriptions  written  for  a  managed  care  population  resulted  in  an 
error. Ernst  and  Grizzle  recently  estimated  the  additional  cost  of  health  care 
resulting  from  medication-induced  morbidity  and  mortality  to  be  $177.4  billion, 
or  about  7  percent  of  total  national  health  expenditures.^  These  large  values  result 
in  part  from  the  volume  of  care  provided  by  physicians  in  ambulatory  care 
settings  and  the  increased  complexity  of  that  care.  As  noted  by  Hammons  et  ah, 
nearly  80  percent  of  all  medical  procedures  are  now  performed  in  ambulatory  care 
settings.^  Moreover,  the  services  provided  during  these  visits  are  becoming  more 
complex  as  enhanced  technologies  and  treatment  modalities  are  transferred  from 
hospital  settings  to  ambulatory  care.^ 

While  medical  group  practices  are  rapidly  becoming  the  practice  form  of 
choice  in  the  ambulatory  setting,  little  is  known  about  what  is,  can,  or  should  be 
done  to  assure  patient  safety  and  quality  of  care.  As  noted  by  Hammons,  a 
physician’s  knowledge,  skills,  and  judgments  influence  patient  care  outcomes,  but 
the  systems  within  which  they  practice  are  factors  as  well.^  In  fact,  the  practice 
organization  may  be  as  important  as  the  clinician’s  skills  in  assuring  quality  of 
care.  Much  of  the  patient  safety  research  has  focused  on  inpatient  care  and  the 
influence  that  internal  hospital  systems  and  procedures  have  on  medical  errors 
and  the  quality  of  care.  This  has  resulted,  in  part,  because  hospital  data  is  often 
more  readily  available  for  study,  and  hospitals  are  often  more  willing  to 
participate  in  studies  than  are  physicians  in  office-based  practices.^ 
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Research  in  medical  group  practices  presents  several  methodological 
challenges  related  to  defining  a  group  practice,  measuring  patient  safety,  and 
synthesizing  the  data  necessary  for  the  estimation.  The  remainder  of  this  paper 
describes  those  methodological  challenges  and  potential  resolutions. 


Specifying  and  measuring  group  practice  variabies 

The  standard  definition  of  medical  group  practices  used  by  the  American 
Medical  Association  and  Medical  Group  Management  Association  (MGMA)  is 
three  or  more  physicians  practicing  together  with  a  common  medical  record  and 
billing  system.^  This  definition  may  not  be  specific  enough  to  support  rigorous 
research  or  even  to  monitor  changes  in  the  field.  Several  additional  attributes  are 
important  defining  characteristics  of  medical  groups,  but  most  are  difficult  to 
assess  from  existing  datasets.  For  example,  one  of  the  most  important  attributes  of 
group  practices  is  the  degree  to  which  the  physicians  consider  themselves  part  of 
an  organized  way  of  providing  health  care,  rather  than  independent  clinicians 
practicing  in  the  same  location  and  sharing  support  services.  This  has  been 
described  as  a  “me  vs.  us”  perspective  about  all  aspects  of  the  group  practice 
organization. Unfortunately,  despite  its  importance,  assessing  it  would  constitute 
a  research  project  in  and  of  itself. 

Then,  how  should  group  practices  be  classified  for  studies  of  patient  safety, 
and  how  can  the  important  organizational  variables  be  specified?  A  useful 
approach  is  to  classify  the  defining  variables  according  to  those  that  can  be 
expected  to  be  reasonably  stable  and  those  that  are  amenable  to  short-term 
changes  by  the  practice  leadership.  Variables  that  have  been  shown  to  be  stable 
over  5-  to  10-year  time  periods  and  influence  practice  performance  include 
location  (urban  vs.  rural),  ownership,  employment  of  nurse  practitioners  or 
physician  assistants,  tax  status  (for  profit  or  not),  decisionmaking  structure,  and 
practice  culture.'^  Attributes  that  are  more  amenable  to  short-term  change  include 
practice  size,  specialty  mix,  degree  of  financial  risk  sharing  for  patient  care, 
physician  compensation  method,  and  clinical  information  system. 

Thus,  we  propose  the  basic  unit  of  analysis  to  be  established  using  the  more 
stable  variables.  Theoretical  models  will  drive  the  inclusion  of  other  attributes. 

For  example,  a  study  of  prescription  drug  errors  in  medical  group  practices  using 
claims  data  from  a  managed  care  plan  can  specify  the  unit  of  analysis  as 
multispecialty  practices  with  three  or  more  physicians  practicing  together,  using  a 
common  billing  system,  and  including  at  least  one  primary  care  physician.  All  of 
these  attributes  are  available  in  the  claims  data.  The  theoretical  model  might  then 
include  variables  such  as  ownership,  urban  location,  physician  compensation 
method,  and  clinical  information  system.  Some  of  these  variables  might  be 
available  from  claims  data,  but  most  would  need  to  be  obtained  from  the 
practices. 

Defining  the  unit  of  analysis  with  respect  to  group  practices  is  not 
straightforward,  and  neither  is  the  specification  of  several  significant  practice 
attributes.  As  mentioned  earlier,  ownership  has  been  shown  to  be  an  important 
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factor  influencing  practice  performanee,  and  espeeially,  aceess  to  eapital.  The 
following  is  the  conventional  approach  to  defining  different  types  of  ownership: 

•  Owned  by  all  of  the  physicians 

•  Owned  by  a  subset  of  the  physieians 

•  Owned  by  a  group  practiee  system 

•  Owned  by  a  hospital  or  hospital  system 

•  Owned  by  a  health  plan  or  health  maintenanee  organization 

•  Owned  by  a  praetiee  management  eompany 

While  this  definition  captures  the  main  influence  of  ownership  on  practiee 
performanee,  three  eautions  should  be  noted.  First,  there  is  a  trend  for  physieian- 
owned  practices  to  be  loeated  in  clinie  faeilities  owned  by  some  other 
organization,  usually  a  hospital  or  health  plan.  When  the  unit  of  analysis  is  the 
group  praetiee,  the  ownership  of  the  practiee,  rather  than  the  elinie,  should  be  the 
eentral  variable.  Clinie  owners,  however,  may  influence  aeeess  to  eapital 
improvements  and  support  services,  and  for  this  reason,  a  separate  variable  should 
address  that  issue. 

A  seeond  ownership  issue  relates  to  the  trend  toward  forming  eonsortia  of 
medical  group  practices  to  share  information  and  negotiate  contracts  with 
managed  eare  plans.  These  umbrella  organizations,  sometimes  ealled  individual 
practiee  assoeiations,  may  own  some  of  the  partieipating  praetiees  and  may  even 
elaim  all  of  them  as  their  elinics.  For  researeh  purposes,  however,  it  is  important 
to  traee  each  practice  to  its  legal  eorporate  ownership  in  order  to  untangle  the 
infiuenee  of  important  explanatory  variables. 

A  third  ownership  issue  relates  to  satellite  elinies.  Often  a  large  group  praetiee 
will  establish  satellite  clinics  to  improve  aceess  to  serviees  and  to  eapture 
additional  referrals.  In  some  eases,  the  parent  group  purehases  existing  praetiees. 
While  the  legal  ownership  in  this  ease  is  elear,  these  satellites  often  operate  as 
separate  entities.  We  have  found  that  responses  to  questions  about  practiee 
attributes  (e.g.,  the  use  of  elinieal  guidelines)  from  satellite  administrators  and 
physieians  differ  from  those  provided  by  the  parent.  Moreover,  the  culture  of 
the  satellite  may  differ  signifieantly  from  that  of  the  parent,  espeeially  if  the 
parent  purehased,  as  opposed  to  established,  the  satellite.  Therefore  it  is  clear  that 
in  some  patient  safety  researeh,  these  satellites  should  be  treated  as  separate  units 
of  analysis.  The  deeision  whether  to  speeify  the  satellites  as  separate  units  of 
analysis  depends  on  the  variables  ineluded  in  the  model  and  the  sourees  of  data.  A 
model  that  ineludes  physician  compensation  methods,  elinieal  information 
systems,  or  use  of  drug  formularies  ean  usually  treat  the  parent  and  the  satellites 
as  one  praetiee  by  assessing  the  degree  to  whieh  the  responses  pertain  to  all  of  the 
elinie  sites.  However,  a  model  that  includes  a  practiee  culture  variable  or  degree 
of  finaneial  risk  sharing  in  the  revenue  streams  may  need  to  treat  each  site  as  a 
unit  of  analysis.  A  useful  signal  from  claims  data  that  helps  resolve  this  issue  is 
the  souree  of  the  bills.  A  satellite  clinic  that  bills  separately  from  the  parent 
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indicates  that  the  clinic  operates  as  a  relatively  autonomous  unit  and  should  be 
treated  as  sueh. 

Finaneial  ineentives  in  group  praetiees  have  been  shown  to  have  an  important 
influence  on  the  costs  and  quality  of  care/^”^’  Group  practice  performance  is 
influeneed  by  both  how  the  praetiee  is  paid  and  how  the  physieians  within  the 
praetiee  are  paid.  While  early  studies  speeified  the  finaneial  ineentive  variable  at 
the  praetiee  level  as  eapitation  or  fee-for-serviee,  this  diehotomy  is  no  longer 
suffieiently  diseerning.  A  useful  alternative  is  to  assess  the  degree  of  finaneial 
risk  sharing  for  patient  eare  in  the  payment  plan.  Using  this  approaeh,  eapitation 
payment  for  all  serviees  defines  one  end  of  the  eontinuum,  and  fee-for-serviee 
based  on  billed  eharges  defines  the  other.  In  between  exist  multiple  arrangements 
sueh  as  eapitation  for  physieian  services  only,  fee-for-serviee  with  withhold 
provisions,  and  negotiated  fee  sehedules.  A  useful  way  to  assess  the  mix  of  these 
payment  plans  is  to  ask  the  praetiee  administration  to  indieate  the  proportion  of 
revenue  derived  from  eaeh  of  the  different  payment  plans  for  the  study  period. 

The  following  eategories  have  been  suceessfully  used  in  several  past  studies:'^’ 

•  Full  eapitation  for  physieian  and  hospital  serviees 

•  Capitation  for  physician  services  only 

•  Capitation  for  primary  eare  physieian  serviees  only 

•  Fee  sehedule  with  target  rates  and  withhold  provisions 

•  Negotiated  discount  from  standard  fee  sehedules 

•  Billed  eharges 

A  useful  method  of  assessing  the  influence  of  physician-level  finaneial 
ineentives  on  group  practice  performanee  is  to  measure  the  degree  to  whieh  the 
physieian’ s  ineome  is  at  risk.  A  predetermined  fixed  salary  earries  the  least  risk, 
while  some  share  of  the  praetiee  net  revenue  has  the  most  risk  of  variation  in 
annual  income.  Moreover,  the  risk  eompensation  plans  may  emphasize  different 
types  of  patient  eare  ineentives.  For  example,  a  plan  that  bases  a  large  portion  of 
the  eompensation  on  some  share  of  net  praetiee  revenue  in  a  largely  eapitated 
praetiee  provides  ineentives  to  reduee  serviee  use.  A  strueture  that  pays  on  a 
diseounted  fee-for-serviee  basis  provides  the  opposite  ineentive.  An  inereasingly 
popular  compensation  plan  in  medical  groups  is  to  base  about  60  pereent  of  the 
physieians’  income  on  some  measure  of  produetivity  and  the  remainder  on  either 
a  fixed  salary  or  a  share  of  net  revenue.  This  provides  the  ineentives  to  work  hard, 
yet  aligns  the  incentives  with  praetiee  goals  imposed  by  the  revenue  streams. 
Physieian-level  payment  eategories  ean  be  speeified  as  the  portion  of 
eompensation  based  on 

•  Fixed,  predetermined  salary 

•  Individual  physieian  produetivity 

•  Share  of  praetiee  net  revenue 

•  Annual  bonus 
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All  of  these  measures  focus  on  the  influence  of  financial  incentives,  rather 
than  level  of  compensation,  on  patient  care  practices.  Level  of  compensation 
clearly  relates  to  practice  styles,  but  is  not  a  driver  of  performance. 

Two  additional  variables  warrant  discussion:  clinical  information  systems  and 
practice  culture.  Both  have  been  shown  to  influence  practice  performance  and 
present  complex  measurement  problems.  A  growing  number  of  medical  groups 
are  adopting  electronic  clinical  information  systems,  and  patient  safety  research 
must  take  this  important  variable  into  account.  There  is  little  agreement,  however, 
on  the  terminology  describing  this  technology.  Consequently,  while  some 
research  focuses  on  the  presence  or  absence  of  an  electronic  medical  record 
(EMR),  the  findings  are  often  suspect  because  of  a  lack  of  agreement  on  the 
definition  of  an  EMR.  ’  The  definition  problem  is  twofold.  Eirst,  some 
practices  consider  an  EMR  to  be  a  computer-based  supplement  to  a  paper  record 
rather  than  a  replacement.  Second,  an  EMR  has  several  components,  and  practices 
may  implement  only  one  or  two  components.  Eor  example,  it  is  not  uncommon 
for  practices  to  have  electronic  patient  information  at  the  care  site,  but  to  write  lab 
orders  and  drug  prescriptions  on  paper.  This  distinction  would  be  very  relevant  to 
studies  of  prescription  drug  errors,  and  a  research  model  should  capture  such 
variations. 

Including  practice  culture  variables  in  analytic  models  presents  a  different 
problem.  While  several  studies  have  found  that  the  culture  of  organizations 
influences  performance,  the  translation  of  that  research  to  the  health  care  field  has 
been  slow  to  develop. Moreover,  there  is  concern  that  the  instruments  used  to 
measure  cultures  in  industrial  firms  may  not  capture  the  nuances  of  organizations 
such  as  medical  group  practices. Researchers  focusing  on  patient  safety  in 
medical  groups  should  be  attentive  to  this  issue  when  selecting  a  culture 
instrument  and  look  carefully  at  the  types  of  organizations  used  to  develop  and 
test  the  instrument.  A  second  caution  is  that  researchers  often  describe 
organizational  culture  using  single-dimension  variables  constructed  as  the  mean 
response  from  the  physicians  and  nurses  to  each  item  on  the  culture  instrument. 
While  this  is  a  valid  first  step  in  assessing  practice  cultures,  the  next  important 
step  is  to  measure  the  degree  of  clinician  agreement  on  the  culture.  Expressed 
together  as  interaction  terms,  these  variables  provide  a  much  more  powerful 
analysis. 

This  discussion  of  group  practice  attributes  is  not  meant  to  be  exhaustive. 
Many  other  factors  may  be  hypothesized  to  have  an  effect  on  patient  safety.  The 
variables  we  discuss  here,  however,  have  been  shown  to  influence  group  practice 
performance  and  are  known  to  present  special  specification  and  measurement 
issues. 


Measuring  patient  safety 

Defining  and  measuring  patient  safety  poses  a  significant  challenge. 
Researchers  have  put  forth  many  definitions  of  patient  safety,  errors,  and  adverse 
events.  We  support  a  definition  of  patient  safety  that  encompasses  care  that  is  not 
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only  free  of  unwanted  consequences,  but  meets  a  given  standard  of  care.  We 
would  like  to  capture  the  idea  that  both  commission  of  inappropriate  procedures 
and  omission  of  appropriate  procedures  can  compromise  patient  safety;  so  for  the 
purposes  of  this  discussion,  let  us  define  patient  safety  as  the  patient  receiving  the 
care  that  was  intended  and  merited.  Patient  care  in  a  group  practice  is  comprised 
of  processes,  even  if  they  are  not  formally  defined  as  such.  Steps  in  these 
processes  can  go  awry  when  an  individual  takes  an  inappropriate  action  or  fails  to 
take  an  appropriate  action.  We  refer  to  these  instances  as  errors.  An  error  may  or 
may  not  result  in  an  adverse  event,  which  is  an  instance  of  a  patient  suffering  the 
consequences  of  an  error.  In  other  words,  errors  may  be  detected  by  monitoring 
the  process,  while  adverse  events  may  be  detected  by  monitoring  the  patient.  It  is 
important  to  note  that  not  all  errors  will  result  in  adverse  events.  Some  health  care 
processes  can  tolerate  a  certain  number  of  errors.  Similarly,  whether  or  not  an 
error  results  in  an  adverse  event  may  be  related  to  patient  characteristics.  For 
these  reasons,  an  analyst  studying  patient  safety  cannot  choose  between  studying 
errors  or  adverse  events — but  must  study  both.  The  importance  of  studying 
adverse  events  is  obvious.  Patients  suffer  when  they  occur.  The  importance  of 
studying  the  errors  that  precede  the  adverse  events  is  also  obvious.  The  errors 
show  how  the  process  failed  the  patient.  What  may  be  less  obvious  is  that  it  is 
also  important  to  study  errors  that  did  not  result  in  adverse  events.  Not  only  are 
they  more  plentiful,  but  they  offer  the  opportunity  to  learn  about  why  the  error  did 
not  result  in  an  adverse  event. 

An  analysis  of  errors  and  adverse  events  thus  requires  an  understanding  of 
both  the  process  of  patient  care  and  patient  outcomes.  Determining  whether  a 
patient  received  the  care  that  was  intended  and  merited  is  not  a  straightforward 
task.  Best  practices  must  be  defined  to  determine  if  safe  care  was  provided. 
Previous  research  has  demonstrated  that  the  standards  of  care  vary  geographically 
and  are  often  different  from  best  practices.  Of  course,  for  many  conditions,  there 
still  exists  uncertainty  about  treatment  options,  especially  when  patient 
preferences  are  taken  into  account.  Fisher  and  Wennberg  address  this  issue  by 
making  the  distinction  between  “effective  care”  and  “price-sensitive  care.”  The 
former  includes  “. .  .only  interventions  that  virtually  all  well-informed  patients 
would  want,  and  that  such  patients  would  expect  physicians  to  recommend  or 
prescribe.”  The  latter  refers  to  “. .  .those  interventions  in  which  there  is  a  choice 
between  at  least  two  treatments  that  have  different  risks  and  benefits,”  from  which 
different  patients  may  draw  different  conclusions. 

A  researcher  commencing  a  patient  safety  study  must  define  the  events  and 
errors  that  might  arise  in  any  given  setting.  Several  studies  have  attempted  to 
identify  and  classify  the  types  of  errors  encountered  in  family  practice.  While 
these  studies  may  serve  as  useful  starting  points,  researchers  should  be  aware  that 
the  definitions  of  errors  and  adverse  events  often  vary  between  studies.  The 
implementation  of  an  error  reporting  system  in  general  practices  in  the  United 
Kingdom  (U.K.)  led  to  a  classification  system  for  the  errors  that  included  the 
categories  prescriptions,  communication,  appointments,  equipment,  clinical  care, 

and  other.  A  survey  of  general  practitioners  from  six  countries  resulted  in  a 

28 

taxonomy  of  errors  they  encountered  in  their  work.  The  study  identified  171 
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types  of  errors  and  eoneluded  that  they  could  be  classified  as  either  process  errors 
or  knowledge/skills  errors.  Family  practitioners  in  the  United  States  reported 
errors  in  the  following  processes:  ordering  medication,  implementing  laboratory 
investigations,  fding,  implementing  medication  orders,  and  responding  to 
abnormal  test  results. They  reported  consequences  of  those  errors  such  as  health 
outcomes  (e.g.,  pain,  worsening  of  condition),  care  consequences  (e.g.,  delay), 
and  financial  and  time  costs. A  review  of  the  few  studies  related  to  errors  in 
primary  care  also  led  to  the  development  of  a  classification  scheme.  Elder  and 
Dovey  distinguished  between  “preventable  adverse  events”  and  “process 

-5  1 

errors.”  In  their  taxonomy,  adverse  events  included  outcomes  related  to 
diagnosis  (e.g.,  missed  diagnosis),  treatment  (e.g.,  incorrect  drug),  and  preventive 
care  (e.g.,  inappropriate).  Note  that  this  definition  of  adverse  events  differs 
significantly  from  ours  in  that  it  does  not  address  health  outcomes.  Process  errors 
in  that  taxonomy  included  factors  related  to  clinicians  (e.g.,  judgment), 
communication,  administration,  and  “blunt  end”  (e.g.,  government  regulations). 

This  discussion  of  errors,  adverse  events  and  standards  of  care  emphasizes 
that  it  is  very  difficult  to  divorce  the  discussion  of  patient  safety  from  the 
discussion  of  health  care  quality.  In  fact,  there  are  disadvantages  to  focusing  on 
errors  and  adverse  events  to  the  exclusion  of  other  quality  indicators: 

A  common  problem  arises  in  the  attempt  to  define  the  performance 
of  a  system  by  a  single  measure  or  a  few  measures  concentrated  in 
one  dimension.  Given  one  measure  of  success,  almost  any  group 
can  be  successful  in  the  short  term  by  optimizing  that  measure  at 
the  expense  of  other  important  measures. 

For  example,  the  Institute  of  Medicine  (lOM)  has  called  for  health  care  to 
improve  in  six  dimensions:  safety,  effectiveness,  timeliness,  patient-centeredness, 
efficiency,  and  equity. The  definition  of  patient  safety  we  have  put  forth 
encompasses  the  lOM’s  criteria  of  safety  and  effectiveness.  There  are  certainly 
aspects  of  timeliness  that  affect  the  safety  of  a  patient,  but  there  are  also  aspects 
that  play  a  role  in  patient  satisfaction.  Finally,  the  remaining  goals  are  not  likely 
to  be  addressed  by  an  emphasis  on  patient  safety  as  we  have  defined  it. 


Data  sources 

Because  of  the  nature  of  both  ambulatory  care  and  patient  safety,  research  in 
this  area  will  likely  involve  data  from  multiple  sources.  As  discussed  earlier, 
errors  and  adverse  events  should  be  measured  by  looking  at  both  processes  and 
patient  outcomes.  In  the  absence  of  a  specially  designed  error  reporting  system, 
these  two  types  of  information  are  unlikely  to  be  found  in  a  single  data  source. 
There  is  a  significant  body  of  research  studying  errors  in  hospitals.  This  setting 
has  the  advantage  of  being  a  (nearly)  closed  system.  If  an  error  occurs  in  a  process 
in  the  hospital,  the  adverse  event  is  likely  to  occur  in  the  hospital.  This  is  not  the 
case  in  ambulatory  care,  where  an  error  that  occurs  in  the  practice  setting  may 
result  in  an  adverse  event  at,  for  example,  the  patient’s  home,  which  then  results 
in  the  patient  going  to  the  emergency  department.  These  points  are  illustrated  by  a 
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recent  study  of  adverse  drug  events  among  elderly  patients  in  an  ambulatory 
setting.  Gurwitz  et  al.  found  it  necessary  to  identify  adverse  events  using  several 
data  sources:  “Reports  from  health  care  providers;  review  of  hospital  discharge 
summaries;  review  of  emergency  department  notes;  computer-generated  signals; 
automated  free-text  review  of  electronic  clinic  notes;  and  review  of  administrative 
incident  reports  concerning  medication  errors.”  In  addition  to  the  challenge  of 
synthesizing  multiple  sources  of  data,  each  type  of  dataset  offers  potential 
advantages  and  challenges.  We  discuss  those  trade-offs  in  the  rest  of  this  section. 

Claims  data 

Claims  submitted  by  medical  group  practices  for  services  provided  to 
enrollees  of  a  health  insurance  plan  offer  a  promising  source  of  large  datasets  at 
low  cost.  These  data  usually  include  physician,  patient,  and  practice  identifiers 
along  with  both  International  Classification  of  Diseases  (ICD)-9  and  Current 
Procedural  Terminology  (CPT)-4  codes  for  services.  These  data  can  provide 
important  process  and,  in  some  cases,  health  outcome  measures  to  assist  in  the 
identification  of  errors  and  adverse  events. 

Claims  data  may,  however,  present  practice  identification  problems  related  to 
those  discussed  earlier.  While  many,  if  not  most,  health  insurance  plans  can 
identify  the  medical  groups  serving  enrollees,  information  about  the  practice 
rarefy  extends  beyond  the  number  and  types  of  physicians  and  the  location.  If  the 
plan  requires  enrollees  to  select  a  practice  to  provide  and  manage  their  care,  costs 
and  quality  measures  can  be  attributed  to  a  specific  group  practice.  If  enrollees  are 
allowed  to  select  any  physician  in  the  plan  for  care  on  an  episodic  basis,  the 
responsible  physician  must  be  determined  from  the  claims.  A  conventional 
method  to  addressing  this  issue  is  to  assign  responsibility  for  prevention  services 
(e.g.,  immunizations,  PAP  smears)  to  the  primary  care  physician  accounting  for 
most  visits  during  the  year,  or  to  the  physician  to  whom  the  last  primary  care  visit 
was  made.  Other  patient  safety  incidents  can  be  traced  to  the  physician 
responsible  for  the  episode  of  care  when  the  incident  occurred  or,  in  the  case  of 
drug  errors,  to  the  physician  who  wrote  the  prescription.  If  the  health  plan 
database  does  not  include  a  practice  identifier,  the  physicians  can  be  placed  in 
their  medical  group  by  the  tax  codes  used  for  claims  billing.  When  verified 
against  health  plan  and  phone  online  directories,  this  approach  has  been  found  to 
place  about  90  percent  of  the  physicians  in  their  practice  organizations.^"^ 

Practice  surveys 

We  have  discussed  the  importance  of  practice  structure  and  culture  to  patient 
safety  research.  While  some  of  this  data  is  available  from  sources  such  as  the 
MGMA,  in  most  cases  the  data  must  be  collected  from  the  group  practices 
themselves.  Several  mechanisms  have  been  used  to  acquire  this  data.  Casalino  et 
al.  successfully  used  phone  interviews  with  group  practice  administrators  and 
physicians  to  obtain  practice  data.  Others  have  combined  phone  interviews  or 
mailed  surveys  with  site  visits.'^  Mailed  or  electronic  questionnaires  are  by  far  the 
least  expensive  methods  to  obtain  group  practice  data,  but  the  response  rates  are 
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often  disappointing.  This  results,  in  part,  from  failure  to  gain  prior  cooperation  of 
the  practice  administrators  in  the  project  and  from  including  too  many  items  in  the 
questionnaires.  Researchers  who  have  successfully  attended  to  these  issues  have 
consistently  obtained  response  rates  in  the  70  to  80  percent  range.  While  most,  if 
not  all,  practice  structure  data  can  be  obtained  from  the  administrator,  the  practice 
culture  data  must  be  obtained  from  the  clinicians.  This  presents  different 
methodological  problems.  Many  researchers  obtain  only  40  percent  response  rates 
to  physician  surveys,  and  there  is  a  great  deal  of  concern  about  potential  biases  in 
the  responses. 

Organizational  culture  studies  risk  serious  flaws  if  the  response  rate  falls 
below  80  percent  because  different  professions  view  the  culture  differently,  and 
response  bias  is  highly  likely  to  have  lower  return  rates.  Various  methods  have 
been  used  to  improve  these  rates,  including  payment  for  completing  the  survey 
instrument  and  holding  raffles  for  trips  to  professional  meetings.  Neither  has 
been  very  successful.  A  method  successfully  used  in  Minnesota  appealed  to  the 
professional  commitments  of  group  practices  to  improve  their  patient  care 
process.  The  group  practice  medical  directors  were  asked  to  distribute  the  culture 
instrument  to  the  physicians  at  one  of  their  routine  meetings,  with  instruments  to 
be  returned  anonymously  to  the  researchers.  This  approach  obtained  high  levels  of 
participation  (84  percent)  while  protecting  the  identity  of  the  respondents.^' 

Patient  surveys 

An  alternate  method  of  acquiring  a  sufficient  amount  of  data  to  support  patient 
safety  research  is  to  interview  patients.  Most  research  using  this  method  focuses 
on  patient  satisfaction  and  access  to  care.  However,  some  studies  have  acquired 
patient  safety  data  using  phone  interviews.  Hanlon  et  al.  found  that  patients  were 
able  to  identify  a  broad  range  of  prescription  drug  errors,  including  the  degree  of 
adverse  effects.  Moreover,  MGMA  has  developed  a  method  of  conducting  group 
practice  patient  interviews  by  phone  while  protecting  the  confidentiality  of  the 
patient,  an  important  consideration  when  using  this  method  of  data  collection. 

Self-reporting  systems 

There  is  growing  recognition  of  the  potential  value  to  health  care  of  incident 
reporting  systems  (both  voluntary  and  mandatory).  ’  For  example,  the  Veterans 
Health  Administration  has  announced  its  intent  to  establish  such  a  system."'*'  The 
American  Academy  of  Family  Physicians  is  conducting  an  evaluation  of  doctor- 
staff-,  and  patient-reported  medical  errors,  which  is  underway  in  five  primary  care 
clinics  and  five  family  physician  offices.""  As  self-reported  error  and  adverse 
event  data  become  increasingly  available,  researchers  should  be  aware  of  the 
incentives  and  disincentives  that  an  individual  may  have  for  reporting  an  error  or 
adverse  event,  and  how  that  may  affect  the  quality  of  the  data.  As  Barach  and 
Small  report  in  their  review  of  nonmedical  “near  miss”  reporting  systems, 
disincentives  to  use  of  a  reporting  system  include  perceptions  of  additional  work, 
fear  of  reprisal,  and  lack  of  trust  in  the  effectiveness  of  the  system  itself. 
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EMRs 

Some  authors  have  suggested  that  EMR  is  the  next  big  development  that  must 
oeeur  to  ensure  safety  and  quality  in  all  health  eare  settings. Researeh  suggests 
that  EMRs  offer  advantages  over  paper  reeords  with  respeet  to  legibility  and 
eompleteness  of  information  related  to  diagnoses,  preseriptions,  adviee  given,  and 
referral."^^  The  implementation  of  EMRs  in  group  praetiees  eould  greatly  expand 
the  kinds  of  patient  safety  researeh  that  is  praetieal  and  may  inerease  the  quality 
of  that  researeh.  An  EMR  eould  obviate  the  need  for  eostly  ehart  reviews,  provide 
a  useful  eounterpoint  to  elaims  data,  and  provide  baekground  information  to 
supplement  self-reported  errors.  We  have  noted  that  the  implementations  of 
EMRs  in  group  praetiees  are  heterogeneous  and  that  patient  safety  entails  looking 
at  both  the  proeess  and  the  outeomes  of  patient  eare.  In  the  ambulatory  eare 
setting,  the  outeomes  ean  oeeur  in  various  loeations  and  at  various  times,  whieh 
may  mean  that  they  oeeur  outside  the  purview  of  the  EMR.  For  this  reason,  the 
EMR  may  be  most  informative  with  respeet  to  the  proeess,  rather  than  the 
outeome  of  patient  eare. 

Ultimately,  rigorous  patient  safety  researeh  requires  multiple  sourees  of  data 
and  approaehes.  The  eosts  of  aequiring  this  data  depend  on  the  degree  to  whieh 
the  praetiee  and  the  external  organizations  have  eleetronie  data  systems  in  plaee 
and  are  willing  to  eollaborate  in  the  researeh. 


Conclusions 

Clearly,  patient  safety  is  and  should  be  an  important  part  of  the  national  health 
eare  agenda.  To  date,  very  little  researeh  has  oeeurred  in  the  ambulatory  eare 
setting,  ineluding  group  praetiees.  Patients  are,  however,  reeeiving  inereasingly 
greater  proportions  of  their  eare  in  these  settings.  For  these  reasons,  we  believe 
that  patient  safety  in  group  praetiees  should  be  a  priority. 

In  this  paper,  we  have  outlined  some  of  the  ehallenges  of  working  in  this  area. 
Group  praetiees  are  heterogeneous  in  nature,  and  we  have  shown  how  that 
presents  speeifieation  and  measurement  ehallenges  and  how  those  ehallenges  may 
be  addressed.  The  definition  and  measurement  of  patient  safety  is  also  a  eomplex 
task.  We  have  ealled  for  a  definition  that  ineludes  errors  and  adverse  events, 
representing  both  the  health  eare  proeess  and  patient  outeomes.  The  eomplex 
nature  of  group  praetiees  introduees  many  estimation  problems.  When  the  data 
represent  multiple  levels  of  strueture,  attributing  effeets  to  the  proper  level  is 
diffieult.  We  have  diseussed  several  approaehes  to  dealing  with  these  ehallenges. 
Finally,  we  deseribed  how  a  eomplex  definition  of  patient  safety  eould  result  in  a 
eomplex  set  of  data  requirements.  Many  potential  data  sourees  exist,  and  eaeh 
presents  its  own  ehallenges.  No  partieular  data  souree,  however,  is  likely  to  be 
suffieient  for  a  rigorous  study  of  errors  and  adverse  events. 
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